The present invention relates to solid state imaging devices. More particularly, the present invention relates to the field of colour imaging sensors.
Solid state image sensors are well known. Virtually all solid-state imaging sensors have as key element a photosensitive element being a photoreceptor, a photo-diode, a photo-transistor, a CCD gate, or the like. Typically, the signal of such a photosensitive element is a current which is proportional to the amount of electromagnetic radiation (light) falling onto the photosensitive element.
A structure with a photosensitive element included in a circuit having accompanying electronics is called a pixel. Such pixel can be arranged in an array of pixels so as to constitute a sensor and to build focal plane arrays.
Commonly such solid state image sensors are implemented in a CCD-technology or in a CMOS- or MOS-technology. Solid state image sensors find a widespread use in devices such as camera systems. In this embodiment a matrix of pixels comprising light sensitive elements constitutes an image sensor, which is mounted in the camera system. The signal of said matrix is measured and multiplexed to a so-called video-signal.
Of the image sensors implemented in a CMOS/MOS-technology, CMOS/MOS image sensors with passive pixels and CMOS/MOS image sensors with active pixels are distinguished. An active pixel is configured with means integrated in the pixel to amplify the charge that is collected on the light sensitive element. Passive pixels do not have such means and require a charge sensitive amplifier that is not integrated in the pixel. For this reason, active pixel image sensors are potentially less sensitive to noise fluctuations than passive pixels. Due to the additional electronics in the active pixel, an active pixel image sensor may be equipped to execute more sophisticated functions, which can be advantageous for the performance of the camera system. Such functions can include filtering, operation at higher speed or operation in more extreme illuminations conditions. Examples of such imaging sensors are disclosed in EP-A-0739039, in EP-A-0632930 and in U.S. Pat. No. 5,608,204.
A problem in the art is to reconstruct a color image from the measured information of the pixel array. In order to obtain a colour image, at least three colour coordinates need to be determined in each pixel. Usually, the pixel colour is specified by its Red, Green and Blue (RGB) intensity. To acquire the RGB colour information, the pixels are covered with Red, Green and Blue (or Yellow, Cyan and Magenta) colour filters, one pixel however having only one colour filter type. Thus the colour image captured by a single-chip camera contains only information about one of the three colour channels for each pixel. The two missing colour values must be estimated or reconstructed in each pixel from the information of the neighbouring pixels. This problem is known as xe2x80x9ccolour filter array interpolationxe2x80x9d. A simple interpolation method takes the missing RGB values for one pixel from the neighbouring pixels. This method may work for uniform regions on large objects. At edges and regions with a lot of details however, the neighbouring pixels could take radiation from other objects. If that information is used for the RGB interpolation, this results in false colours. For example, when a red pixel (a pixel covered with a red colour filter) detects a black object, while its green and blue neighbour pixels detect a white background, the red pixel is interpolated as a cyan (green+blue) point instead of white.
Examples of xe2x80x9ccolour filter array interpolationxe2x80x9d techniques are disclosed in the publications:
xe2x80x9cInteractions between colour plane interpolation and other image processing functions in electronic photographyxe2x80x9d by J. E. Adams, in SPIE 2416 p 144 (1995); and
xe2x80x9cA New Digital Signal Processor for Progressive Scan CCDxe2x80x9d by H. Zen et al. in IEEE Trans. Cons. Electr. 44, p 289 (1998).
However, the prior art fails to disclose a colour filter interpolation technique that delivers a performing image quality and at the same time is implementable in a sufficiently small electronic system.
The present invention aims to disclose a colour filter interpolation technique that delivers a performing image quality and at the same time is implementable in a sufficiently small circuit allowing for the construction of a single chip CMOS based colour imaging device.
In a first object of the invention, a method is disclosed for determining at least two types of corrected aspect values for an aspect of a pixel, said pixel being embedded in a configuration of pixels, the method comprising the step of measuring at least one intensity value corresponding to said aspect for essentially each of the pixels of said configuration of pixels. The configuration of pixels can be part of a sensor for making up images that are reconstructed with the information obtained by a measurement on the pixels. Said aspect of said pixels can be the colour of said pixels and the types of corrected colour values can be the Red, Green and Blue colour values of said pixel. The pixels of said configuration in such embodiment of the invention typically each have one colour filter (for instance Red, Green or Blue) that is physically present upon or adjacent to or abutting the pixel. Therefore the pixels, after an intensity measurement, typically each have one measured intensity value that corresponds to said colour aspect of said pixel as the measured intensity value is influenced by the specific colour filter upon the pixel.
The method of the invention further comprises the steps of determining a first corrected intensity value for said pixel by combining the measured intensity values of a set of pixels out of said configuration of pixels; analysing the measured intensity value of said pixel in said corrected intensity value and a subaspect value; and thereafter determining at least two types of corrected subaspect values for said pixel by combining the subaspect values of a set of pixels out of said configuration of pixels.
The method of the invention can further comprise the steps of determining a second corrected intensity value for said pixel by combining said first corrected intensity value with the corrected subaspect value for said pixel obtained by analysing the measured intensity value in said first corrected intensity value and a subaspect value.
The method can also comprise the step of determining at least two types of corrected aspect values for said aspect of said pixel by combining said second corrected intensity value with two types corrected subaspect values for said pixel. Thus in this way a corrected intensity value is determined with the full resolution of the sensor.
As stated above, said pixel is embedded in a configuration of pixels, the pixels of said configuration of pixels can have at least one measured intensity value corresponding to said aspect. Said aspect of said pixels can be the colour of said pixels and the corrected aspect values of said colour can be the Red, Green and Blue colour values of said pixel.
Said subaspect can be the chrominance of said pixel. The corrected values of said colour are then the Red, Green an Blue value of said pixel and the corrected values of said chrominance are the red, green and blue chrominances of said pixel. The sets of pixels preferably are selected within immediate neighbourhood of, also referred to as a window around, said pixel. Such window or immediate neighbourhood can consist of less than about the hundred or fifty closest neighbour pixels of said pixel or of less than about the twenty or ten closest neighbour pixels in the same row as said pixel. Thus the method of the invention can be about determining at least two corrected colour values for a pixel, said pixel being embedded in a configuration of pixels and having a colour filter for filtering substantially one colour type while obtaining a measurement on the pixel, said method comprising the steps of measuring at least one signal on said pixel; transforming the measured signal into a representation having at least a luminance and a chrominance part; and transforming said representation into a colour space representation of said pixel, said pixel having at least two colour values in said colour space representation. The chrominance is corresponding to a colour.
The method can also comprise the steps of selecting out of said configuration a set of pixels in a region around said pixel; selecting out of said set of pixels a subset of pixels having said one colour; and evaluating the value of said colour for essentially each pixel of said subset of pixels.
The terms combining and analysing as referred to hereabove and further in the specification include operations such as adding and multiplying, respectively subtracting and dividing, and other calculator operations such as vote-operations and median-operations and combinations of all these operations (e.g. making an average by adding a number of values and dividing the final added value by the number). The term aspect of a pixel is meant to mean a characteristic parameter of the pixel that can have an impact on the image or image quality of an image that is made with information measured on the pixel. Values for aspects of pixels can be obtained by measurement or by operation on measured values of the pixel. A pixel can have as an aspect the colour of the pixel. The colour of a pixel in the art typically is constructed as a combination of the Red, Green and Blue colour values of the pixel, or of the Cyan, Magenta and Yellow colour values of the pixel. The intensity or luminance or grey value of a pixel is a measured value being correlated to the intensity of the radiation impinging on the pixel, the differences in intensity of the radiation on the different pixels of the configuration of pixels determining the image that can be obtained with the configuration of pixels. The intensity or luminance or grey value of a pixel is the component of the measured data that does not include any colour information. The chrominance of a pixel is the set of values being correlated to the colour (hue and saturation) values of the pixel. The chrominance is the component of the measured data or signals on the pixel that does not contain any intensity information. A colour can be characterized by its luminance and two chrominances. Two colours having the same set of luminance and chrominance data appear to be the same for a human observer. This principle is called Metamerism. The term corrected value as used throughout this patent application can mean any changed value such as an adapted value or an estimated value or any other kind of changed value.
In a second object of the invention, an electronic system is disclosed for implementing in hardware any of the methods recited above. In a preferred embodiment an integrated circuit, preferably a silicon chip, is disclosed integrating a configuration of pixels within an image sensor and integrating a colour filter interpolation technique, each pixel having a colour filter for filtering substantially one colour type, said chip comprising a finite-impulse-response filter having a delay line for storing of the pixels in the immediate neighbourhood of each pixel of said configuration the measured intensity values corresponding to the colour filters of said pixels.